The present invention relates to the presentation of images and, more particularly, to the presentation of successive images.
Still photography, motion pictures and television were influenced by the way artists represented physical reality in paintings, as if through a window. A highly detailed perspective image is provided, typically within a rectangular frame. All provide highly detailed images which induce the viewer to cooperate with the cameraman""s xe2x80x9cvisionxe2x80x9d by assuming the artificial perspective of the representation. The viewer is to some extent enabled to deliberately suspend disbelief that the images themselves are not a real object space. The degree to which the viewer is thus enabled is influenced not only by the image resolution but by the field of view. It is usually thought desirable to increase both. For example, very high resolution commercial television standards have been formulated for increasing image quality. Such approaches typically increase the number of horizontal lines scanned to a number significantly greater than present standards. Larger format movie film such as 70 mm has been used to increase detail. Also, panoramic movies, e.g., xe2x80x9cCineramaxe2x80x9d increased the field of view to increase realism. Various stereoscopic television approaches have also been conceived or developed to increase realism.
All of these traditional media take a rather objective view of the physical world. The image is framed by a window through which the viewer can gaze in any direction xe2x80x9cintoxe2x80x9d a representation of an object space. Events are presented in both movies and television in a series of different action scenes in a story line which the viewer can observe from a seemingly quasi-omniscient point of view. The viewer is led to take what appears to be a view of the world as it really is. Yet the choice of image and its perspective is picked by the creator of the image and the viewer actually assumes a passive role.
A sensorama simulator was disclosed by Heilig in U.S. Pat. No. 3,050,870. The senses of an individual were stimulated to simulate an actual experience realistically with images, a breeze, odors, binaural sound and even motion. Heilig also disclosed a stereoscopic television in U.S. Pat. No. 2,955,156. This also was passive.
xe2x80x9cVirtual reality,xe2x80x9d in an electronic image context, goes even further in the direction of increased realism but enables the viewer to take a much more active role in selecting the image and even the perspective. It means allowing a viewer""s natural gestures, i.e., head and body movements, by means of a computer, to control the imaged surroundings, as if the viewer were seeing and even moving about in a real environment of seeing, hearing and touching. The object space needs to be modelled in its entirety and the position and gestures of the viewer monitored. As the viewer moves about in the virtual object space, the perspective for viewing the virtual objects must be changed by tranformation of coordinates to correspond to the viewer""s position and perspective. This would represent the ultimate in artificial experience if the objects were touchable. The xe2x80x9cobjectsxe2x80x9d in such a virtual environment are not real, however, since they are xe2x80x9ccreatedxe2x80x9d as xe2x80x9cvirtual objectsxe2x80x9d in the database or xe2x80x9creality engine.xe2x80x9d
Due to the myriad of possible actions of the viewer, a corresponding multiplicity of virtual scenarios needs to be available from the reality engine for viewer choice. The creation of many possible scenarios for viewer selection creates a problem of massive demand for image storage space.
There is also the problem of a disconcerting time lag between the viewer""s action and the response of the imaging system. These problems make this emerging technology hard to achieve using presently available hardware.
A contact-analog headup display disclosed in U.S. Pat. No. 5,072,218 showed symbolic images superimposed at selected points on a pilot""s view of the earth as the aircraft overflies the earth. The position and attitude of the aircraft with respect to the earth and the attitude of the helmet with respect to the aircraft are monitored in order to convert a plurality of stored earth position signals into helmet coordinates. Selected points on earth, such as flightplan waypoints, viewable through the visor of the headup display by the pilot, have symbolic flags planted thereon by means of the display, i.e., the waypoint symbols remain xe2x80x9cstuckxe2x80x9d on the earth, in the eyes of the pilot, regardless of the attitude of the aircraft and regardless of the attitude of the helmet. The pilot of course can distinguish the unreal symbolic images from the real environment, i.e., the earth.
The object of the present invention is to provide a new method and means of presenting images in succession.
According to the present invention, xe2x80x9cactualxe2x80x9d images of actual objects in an object space, taken from a moving perspective of a viewer in the object space, are combined with retrieved images imitative of stationary or moving xe2x80x9cvirtualxe2x80x9d objects in the object space for providing the combined images to the viewer as if the retrieved images were also actual.
The idea is to provide images of the virtual objects such that they are indistinguishable from the images of the actual objects. In other words, the idea is to make it hard to tell the difference or make it difficult to distinguish the real environment from the unreal. Since actual objects in the object space are just re-imaged, from the point of view of the viewer, i.e., the images of the actual objects are in registration with the actual objects, they may be touched, used, navigated around, over and under and even moved about by the viewer without difficulty. Virtual objects may xe2x80x9centerxe2x80x9d the imaged object space as if part of the scene and the viewer may interact with these virtual objects according to various alternatives in preprogrammed scenarios.
In further accord with the first aspect of the present invention, an imaging system is provided for use in an object space equipped with prearranged real objects which have their positions and geometric features prestored in a computer spatial model of the object space; the system includes a head mounted display having at least one camera or, alternatively, two cameras for stereoscopic display of images the real objects for a viewer in the object space, wherein the system also includes means for monitoring the position of the head mounted display in the object space and its attitude on the viewer""s head with respect to the object space, and wherein, the system further includes virtual object image strorage means for providing images thereof retrieved from an image storage medium and provided on the display, depending on the viewer""s movements within the object space, and wherein the retrieved virtual object images are combined with the real object images so as to present an integrated whole. To reduce the imaging burden on the reality engine, most of the image content of the scenario may represent real objects and be obtained from the camera or cameras mounted on the viewer""s head. Since the viewer""s position and the helmet""s attitude in the object space are monitored, the virtual object images may be provided in registrated relation to the object space. A seamless integration may be achieved by occluding the portion of the real object images taken up by the virtual object image part retrieved from storage.
The position of the viewer in the object space may be tracked in any number of known methods from the highly sophisticated, such as inertial navigation, to the relatively crude, such as an altimeter in conjunction with floor pressure sensors. The attitude of the head of the viewer may be monitored in a sophisticated manner by means of three-axis gyros or relatively crudely by means of 3-axis inclinometers, for example. It will be understood that other equivalent methods are within the skill of the art.
In still further accord with the present invention, in addition to monitoring the attitude of the head of the viewer with its three rotational degrees of freedom, at least one and preferably two of the three rotational degrees of freedom of at least one of the viewer""s eyes are monitored for the purpose of increasing realism by directing most of the computational power of the imaging system at the portion of the image to which the viewer""s gaze is directed, which may be in the area of imaging of actual objects or persons or virtual objects or persons.
The present invention, by permitting the reality engine to focus almost exclusively on the imaging of virtual objects, permits a virtual reality that greatly reduces the image storage burden on the reality engine. Furthermore, by imaging a real object space directly, images can be provided almost instantaneously, without the burdensome object space image creation processing and the disconcerting time lag of the prior art disappears. Only the images of virtual objects need to be processed, and since they take up a relatively small role in the total image space, the computational power of the image processing system can be fully utilized to speed up the imaging of the virtual objects. And the real objects may be touched.
These and other objects, features and advantages of the present invention will become more apparent in light of the detailed description of a best mode embodiment thereof, as illustrated in the accompanying drawing.